Benchmarking performance indices: pitfalls and solutions

Many organizations include benchmarking as a component of their performance management system. Often, a performance index is used to quantify the ability of an organizational entity to operate successfully. Benchmarking a performance index is done inappropriately when statistical methods are employed that ignore sample size effects or use aggregate performance data over a period during which changes occurred within the organization. Benchmarking will also be ineffective when invalid targets are employed. When benchmarking is done incorrectly, customer satisfactionmay actually decline due to gaming and poor morale among employees. Based on the philosophy of W. Edwards Deming, the techniques of statistical process control (SPC), and basic undergraduate statistics, a system is described for effectively benchmarking a performance index. Examples are presented to illustrate the pitfalls that exist in many performance management systems and to explain the system presented for effective benchmarking. Introduction Many organizations address some form of a benchmarking process as an integral part of their performance management system. Such benchmarking is often performed based on internal quality system requirements (e.g. six-sigma or total quality management (TQM)). Benchmarking may be used, for example, to identify those manufacturing cells that achieve consistently higher yields than cells making similar products, or to identify call center operators who perform better or worse than their peers. External requirements also motivate the need for the development of a performance benchmarking system. These motivations may stem from competitive pressures or from requirements imposed by certi®cation or accreditation requirements, governmental regulations, or customers. For example, in the USA, The Joint Commission on Accreditation of Healthcare Organizations (JCAHO) requires a benchmarking process for healthcare facilities. With the exception of manufacturing entities where measurement data are generally used to analyze performance to design or customer speci®cations, performance indices in the form of a ratio or proportion are very common in most non-manufacturing organizations. For example, the technology employed by call centers may automatically record, for each call, the abandonment rate of calls received, expressed as the ratio of the number of callers that hung up while on hold divided by the number of callers. A hospital may record the mortality rate for a certain disease category (ratio of the number of deaths to the number of patients). In manufacturing, this type of performance index is The Emerald Research Register for this journal is available at The current issue and full text archive of this journal is available at http://www.emeraldinsight.com/researchregister http://www.emeraldinsight.com/1463-5771.htm Benchmarking performance indices